1. Field of Invention
This invention relates to systems and methods of operation for delivery of multi-media content including hotspots to end users. More particularly, the invention relates to systems and methods for unifying hotspots subject to non-linear transformation and interpolation between heterogeneous media representations.
2. Background Discussion
In recent years, there has been a sizable growth in the use of Rich Media over the Worldwide Web as more and more individuals and institutions begin to realize the Web""s potential and broad range of applications, including electronic commerce, education, training, news, etc. Examples of Rich Media include, inter alia, animation, audio, 3-D, panoramas and videos. There are two apparent clusters of media in Rich Media technology. One at the low end and the other at the high end which hamper effective deployment of some Rich Media in Internet based applications. A xe2x80x9clow endxe2x80x9d cluster comprises static images and simple non-interactive animation (typically animatedxe2x80x94Graphic Interface Formats (GIFs) which are easy to deploy and therefore have widespread usage. The high end cluster comprises richer and more natural experiences with larger interactivity, such as panoramas, 3-D, streaming audio/video and composite media (e.g., MPEG-4). The difficulty of deployment has limited widespread usage of Rich Media. A novel scalable architecture called HotMedia bridges the gap between the two clusters thereby achieving widespread WEB penetration. Hot Media is described in IBM Research Report RC21519 entitled xe2x80x9cThe Hot Media Architecture:Progressive and Interactive Rich Media For The Internetxe2x80x9d by K. G. Kumar et al. published July, 1999 A key feature of HotMedia is a suitable delivery file format that contains heterogeneous compositions of media bit streams as well as meta data that define behavior, composition and interaction semantics. The delivery file format enables creation of light weight single file representation of interactive, multi-stage presentations resulting in multiple media type content. At the core of a HotMedia client is a smart content algorithm that infers media type from the incoming data stream and fetches the media renderer components, user-interface components and hyper-linked action components, all just in time, resulting in progressive and context driven enrichment of the user experience. Details related to HotMedia architecture are described in Ser. No. 09/268,537 entitled xe2x80x9cFramework for Progressive Hierarchical and Adaptive Delivery Rich and Media Presentation and Associated Dataxe2x80x9d, supra.
Often, instances of Rich Media incorporate xe2x80x9chotspotsxe2x80x9d to other presentations to expand a user""s experience. The process of clicking on a hotspot in a media for transfer to the another presentation is also referred to as xe2x80x9chotlinkingxe2x80x9d or xe2x80x9chyperlinkingxe2x80x9d which is further described in U.S. Pat. No. 5,819,78 entitled xe2x80x9cNetworking Using Stegnographically Embedded Data Objectxe2x80x9d issued Nov. 24, 1198 and U.S. Pat. No. 5,918,012 entitled xe2x80x9cHyperlinking Time Based Dataxe2x80x9d issued Jan. 29, 1999.
Prior art related to mechanisms for accessing hotspot include the IBM PanoramIX system, which displays a hotspot by coloring a low resolution version of a panorama with pixels of a special color. During display of the projected image at the client station, those image pixels that happen to hover over the specially colored pixels in the secondary panoramic image are modified to show that they represent a hotspot. As a result, hotspot areas may be an arbitrary shape at the expense of a size and logistical problems of maintaining the second image. Another mechanism for accessing hotspot includes Interaction Pictures Incorporated (IPIX) in which a xe2x80x9cclick herexe2x80x9d icon on view window over the hotspot masks the exact shape of the pick or click area.
One problem in xe2x80x9chotlinkingxe2x80x9d between heterogeneous media involves unifying hotspots in different forms of Rich Media. Depending upon the media, some hotspots transform linearly from one media to another media. In other cases, hotspots transform non-linearly from one media to another media.
Ordinarily, media project onto a display screen with scaling, translation, shear and rectilinear projection, which causes straight lines in the media to map to straight lines in the display. Linear translation allows hotspots (where xe2x80x9cclickingxe2x80x9d or xe2x80x9cpickingxe2x80x9d on a hotspot using a mouse to cause action) to be defined by an outline of the two points connected by a straight line.
However, for some media, e.g., panoramas, the mapping from the media to the display is non-linear and the straight lines become curved after transformation. In such media, reverse transformation from the display to the media also creates curved lines. For these media, hotspot outlines cannot be defined by widely separated points, because interpolating straight lines between the points and the media space will cover a slightly different area than interpolating straight lines between points in the display space.
The variety of transformations for media to be displayed and re-stored are a problem for an action handler associated with a display for responding to a xe2x80x9cpickedxe2x80x9d hotspot. Action handlers in the client station perform the transformations between arbitrary spaces. However, non-linear media can be arbitrary curved surfaces (e.g., cylinders or spheres). It is unreasonable for a general-purpose action handler to account for all possible media non-linearities and display options. An action handler must have the following characteristic to overcome problems in unifying hotspot locations between heterogeneous media representation:
(1) Store hotspot (where clicking or picking takes place) separate from the media.
(2) Store hotspot in a way independent of non-linearities of a media.
(3) Detect if picks occur on the hotspot even though the media may have non-linear distortion; and
(4) Provide means for non-linear media to communicate a uniform storage method in the action handler.
Providing a systems and methods to unify hotspots subject to non-linear transformation and interpolation in heterogeneous media representations will advance the growth and the utility of Rich Media for a wide variety of applications in the World Wide Web.
An object of the invention is a system and method for translating hotspots from one media to another media without significant alteration of hotspot locations in both media.
Another object is a system and method for unifying hotspots subject to linear and non-linear transformation in heterogeneous media representations.
Another object is a network and method of operation for transforming coordinates for hotspots in mixed media to corresponding coordinates for hotspots in another media.
Another object is an action handler and a network implementing HotMedia Architecture to store hotspots separate from the media and independent of non-linearity of the media.
Another object is an action handler and a network implementing HotMedia Architecture a which detects that a hotspot is clicked or picked even though the media may have non-linear distortions.
Another object is an action handler and a network implementing HotMedia Architecture which enables non-linear media to communicate coordinates in a uniform storage method.
Another object is an action handler and a network implementing HotMedia Architecture in which hotspot are stored in a Hotspot coordinate system for media programs.
Another object is an action handler and a network implementing HotMedia Architecture in which display and picking in an image is converted to a hotspot coordinate system for media programs before sending queries to the action handler.
These and other object features and advantages are achieved in a system including a content creator and delivery station coupled to a network for delivery of Rich Media presentations to at least one client station, including presentation and user interaction logic in a client terminal. The presentation and user interaction logic includes a client master control module coupled to at least one action handler. The content creator creates media, for example, stills, animation, and panorama including hotspots for delivery to the client station and display as a projected image for user interaction with the hotspot. The hotspot(s) is expressed in the media as an area defined in terms of a new Euclidean rectangular coordinate system i.e., a hotspot coordinate system. The new hotspot coordinate system is used by the master control module in the transformation and mapping of the media image hotspot coordinates into the new corresponding coordinates of a hotspot location in a projected image in a display for user interaction. In some cases, the image coordinates and the media do not necessarily map directly into corresponding coordinates in the projected image. Flat media can be linearly transformed to a projected image coordinates. However, panorama media require non-linear transformation of media coordinates to project an image coordinates in the display. The coordinates for the media hotspot in the hotspot coordinate system are stored in a centralized action handler independent of the non-linearities in the media. Media non-linearities and display options are handled by the media programmer via four specified functions at the media level outside of the action handler The division of labor devised between the action handler and the media programmer is key to how a general-purpose action handler can deal with arbitrary media mappings for picking and display and also deal with media discontinuities and singularities. A user selecting or picking a hotspot in the projected image at the client station may cause the pick to fall outside of the corresponding hotspot for non-linear media delivered by the content creator station to the client terminal. The hotspot in the media and the projected image may be unified by increasing the number of points in the hotspot coordinate system area geometry defining the hotspot in the media. Increasing the number of points in the hotspot geometry makes more reliable that a pick will occur in a projected image and the hotspot will fall within the hotspot area in the media. When a projected image hotspot is picked by a mouse or other input device, a signal is transmitted to the media indicating coordinates of the input device with respect to the media. The media transforms the input coordinates into the hotspot coordinate system and queries the action handler for the storage of the hotspot at the location of the image device. The action handler returns the hotspot, if any, at the query location. The media displays the hotspot, if any, at the location of the input device location on the media. When the pick falls within the media hotspot area and is actuated, the action handler initiates the events associated with the picked hotspot in the projected image. When the pick falls outside of the media hotspot area, the action handler does not initiate an event. Converting the image coordinates to the hotspot coordinate system makes an action handler responding to picking more reliable regardless of the media form of the Rich Media presentation.